Updating object motion dynamics from multiple radio signals

ABSTRACT

In one embodiment, a service receives signal characteristic data indicative of characteristics of wireless signals received by one or more antennas located in a particular area. The service uses the received signal characteristic data as input to a Bayesian inference model to predict physical states of an object located in the particular area. A physical state of the object is indicative of at least one of: a mass, a velocity, an acceleration, a surface area, or a location of the object. The service updates the Bayesian inference model based in part on the predicted state of the object and a change in the received signal characteristic data and based in part by enforcing Newtonian motion dynamics on the predicted physical states.

RELATED APPLICATION

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/857,940, filed on Jun. 6, 2019, entitled “UPDATING OBJECT MOTIONDYNAMICS FROM MULTIPLE RADIO SIGNALS” by Maluf et al., the contents ofwhich are incorporated by reference herein.

TECHNICAL FIELD

The present disclosure relates generally to computer networks, and, moreparticularly, to updating object motion dynamics from multiple radiosignals.

BACKGROUND

The problem of inferring an object model and the behavior of an objectwith a high resolution is not an easy task when doing so must beaccomplished in an efficient, real time, and comprehensive manner.Indeed, such an undertaking requires near optimal integration of all ofthe available data from any number of data sources. More specifically,with the proliferation of wireless communications throughout the world,it now becomes possible to leverage the radio frequency (RF)communication signals to infer a model of the corresponding object. Forexample, a slight variance in the phase shift of a wireless signal, withno change in the attenuation of the signal, can be used to infer thatthe object is in some sort of motion. However, challenges still remainto translate the wireless signals into a usable object model that can beused for purposes of object recognition, delivering services to theobject, etc.

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments herein may be better understood by referring to thefollowing description in conjunction with the accompanying drawings inwhich like reference numerals indicate identically or functionallysimilar elements, of which:

FIG. 1 illustrate an example computer network;

FIG. 2 illustrates an example network device/node;

FIG. 3 illustrates an example wireless network;

FIG. 4 illustrates an example of representing a human as a center ofmass;

FIG. 5 illustrates an example of representing a human using kinematics;

FIG. 6 illustrates an example of transmitting wireless signals towards ahuman;

FIGS. 7A-7C illustrate examples of wireless signals;

FIG. 8 illustrates an example Bayesian inference framework;

FIGS. 9A-9C illustrate example plots of test results; and

FIG. 10 illustrates an example simplified procedure for updating objectmotion dynamics based on the characteristics of received wirelesssignals.

DESCRIPTION OF EXAMPLE EMBODIMENTS Overview

According to one or more embodiments of the disclosure, a servicereceives signal characteristic data indicative of characteristics ofwireless signals received by one or more antennas located in aparticular area. The service uses the received signal characteristicdata as input to a Bayesian inference model to predict physical statesof an object located in the particular area. A physical state of theobject is indicative of at least one of: a mass, a velocity, anacceleration, a surface area, or a location of the object. The serviceupdates the Bayesian inference model based in part on the predictedstate of the object and a change in the received signal characteristicdata and based in part by enforcing Newtonian motion dynamics on thepredicted physical states.

DESCRIPTION

A computer network is a geographically distributed collection of nodesinterconnected by communication links and segments for transporting databetween end nodes, such as personal computers and workstations, or otherdevices, such as sensors, etc. Many types of networks are available,ranging from local area networks (LANs) to wide area networks (WANs).LANs typically connect the nodes over dedicated private communicationslinks located in the same general physical location, such as a buildingor campus. WANs, on the other hand, typically connect geographicallydispersed nodes over long-distance communications links, such as commoncarrier telephone lines, optical lightpaths, synchronous opticalnetworks (SONET), synchronous digital hierarchy (SDH) links, orPowerline Communications (PLC), and others. Other types of networks,such as field area networks (FANs), neighborhood area networks (NANs),personal area networks (PANs), etc. may also make up the components ofany given computer network.

In various embodiments, computer networks may include an Internet ofThings network. Loosely, the term “Internet of Things” or “IoT” (or“Internet of Everything” or “IoE”) refers to uniquely identifiableobjects (things) and their virtual representations in a network-basedarchitecture. In particular, the IoT involves the ability to connectmore than just computers and communications devices, but rather theability to connect “objects” in general, such as lights, appliances,vehicles, heating, ventilating, and air-conditioning (HVAC), windows andwindow shades and blinds, doors, locks, etc. The “Internet of Things”thus generally refers to the interconnection of objects (e.g., smartobjects), such as sensors and actuators, over a computer network (e.g.,via IP), which may be the public Internet or a private network.

Often, IoT networks operate within a shared-media mesh networks, such aswireless or PLC networks, etc., and are often on what is referred to asLow-Power and Lossy Networks (LLNs), which are a class of network inwhich both the routers and their interconnect are constrained. That is,LLN devices/routers typically operate with constraints, e.g., processingpower, memory, and/or energy (battery), and their interconnects arecharacterized by, illustratively, high loss rates, low data rates,and/or instability. IoT networks are comprised of anything from a fewdozen to thousands or even millions of devices, and supportpoint-to-point traffic (between devices inside the network),point-to-multipoint traffic (from a central control point such as a rootnode to a subset of devices inside the network), and multipoint-to-pointtraffic (from devices inside the network towards a central controlpoint).

Fog computing is a distributed approach of cloud implementation thatacts as an intermediate layer from local networks (e.g., IoT networks)to the cloud (e.g., centralized and/or shared resources, as will beunderstood by those skilled in the art). That is, generally, fogcomputing entails using devices at the network edge to provideapplication services, including computation, networking, and storage, tothe local nodes in the network, in contrast to cloud-based approachesthat rely on remote data centers/cloud environments for the services. Tothis end, a fog node is a functional node that is deployed close to fogendpoints to provide computing, storage, and networking resources andservices. Multiple fog nodes organized or configured together form a fogsystem, to implement a particular solution. Fog nodes and fog systemscan have the same or complementary capabilities, in variousimplementations. That is, each individual fog node does not have toimplement the entire spectrum of capabilities. Instead, the fogcapabilities may be distributed across multiple fog nodes and systems,which may collaborate to help each other to provide the desiredservices. In other words, a fog system can include any number ofvirtualized services and/or data stores that are spread across thedistributed fog nodes. This may include a master-slave configuration,publish-subscribe configuration, or peer-to-peer configuration.

Low power and Lossy Networks (LLNs), e.g., certain sensor networks, maybe used in a myriad of applications such as for “Smart Grid” and “SmartCities.” A number of challenges in LLNs have been presented, such as:

1) Links are generally lossy, such that a Packet Delivery Rate/Ratio(PDR) can dramatically vary due to various sources of interferences,e.g., considerably affecting the bit error rate (BER);

2) Links are generally low bandwidth, such that control plane trafficmust generally be bounded and negligible compared to the low rate datatraffic;

3) There are a number of use cases that require specifying a set of linkand node metrics, some of them being dynamic, thus requiring specificsmoothing functions to avoid routing instability, considerably drainingbandwidth and energy;

4) Constraint-routing may be required by some applications, e.g., toestablish routing paths that will avoid non-encrypted links, nodesrunning low on energy, etc.;

5) Scale of the networks may become very large, e.g., on the order ofseveral thousands to millions of nodes; and

6) Nodes may be constrained with a low memory, a reduced processingcapability, a low power supply (e.g., battery).

In other words, LLNs are a class of network in which both the routersand their interconnect are constrained: LLN routers typically operatewith constraints, e.g., processing power, memory, and/or energy(battery), and their interconnects are characterized by, illustratively,high loss rates, low data rates, and/or instability. LLNs are comprisedof anything from a few dozen and up to thousands or even millions of LLNrouters, and support point-to-point traffic (between devices inside theLLN), point-to-multipoint traffic (from a central control point to asubset of devices inside the LLN) and multipoint-to-point traffic (fromdevices inside the LLN towards a central control point).

An example implementation of LLNs is an “Internet of Things” network.Loosely, the term “Internet of Things” or “IoT” may be used by those inthe art to refer to uniquely identifiable objects (things) and theirvirtual representations in a network-based architecture. In particular,the next frontier in the evolution of the Internet is the ability toconnect more than just computers and communications devices, but ratherthe ability to connect “objects” in general, such as lights, appliances,vehicles, HVAC (heating, ventilating, and air-conditioning), windows andwindow shades and blinds, doors, locks, etc. The “Internet of Things”thus generally refers to the interconnection of objects (e.g., smartobjects), such as sensors and actuators, over a computer network (e.g.,IP), which may be the Public Internet or a private network. Such deviceshave been used in the industry for decades, usually in the form ofnon-IP or proprietary protocols that are connected to IP networks by wayof protocol translation gateways. With the emergence of a myriad ofapplications, such as the smart grid advanced metering infrastructure(AMI), smart cities, and building and industrial automation, and cars(e.g., that can interconnect millions of objects for sensing things likepower quality, tire pressure, and temperature and that can actuateengines and lights), it has been of the utmost importance to extend theIP protocol suite for these networks.

FIG. 1 is a schematic block diagram of an example simplified computernetwork 100 illustratively comprising nodes/devices at various levels ofthe network, interconnected by various methods of communication. Forinstance, the links may be wired links or shared media (e.g., wirelesslinks, PLC links, etc.) where certain nodes, such as, e.g., routers,sensors, computers, etc., may be in communication with other devices,e.g., based on connectivity, distance, signal strength, currentoperational status, location, etc.

Specifically, as shown in the example network 100, three illustrativelayers are shown, namely the cloud 110, fog 120, and IoT device 130.Illustratively, the cloud 110 may comprise general connectivity via theInternet 112, and may contain one or more datacenters 114 with one ormore centralized servers 116 or other devices, as will be appreciated bythose skilled in the art. Within the fog layer 120, various fognodes/devices 122 (e.g., with fog modules, described below) may executevarious fog computing resources on network edge devices, as opposed todatacenter/cloud-based servers or on the endpoint nodes 132 themselvesof the IoT layer 130. Data packets (e.g., traffic and/or messages sentbetween the devices/nodes) may be exchanged among the nodes/devices ofthe computer network 100 using predefined network communicationprotocols such as certain known wired protocols, wireless protocols, PLCprotocols, or other shared-media protocols where appropriate. In thiscontext, a protocol consists of a set of rules defining how the nodesinteract with each other.

Those skilled in the art will understand that any number of nodes,devices, links, etc. may be used in the computer network, and that theview shown herein is for simplicity. Also, those skilled in the art willfurther understand that while the network is shown in a certainorientation, the network 100 is merely an example illustration that isnot meant to limit the disclosure.

Data packets (e.g., traffic and/or messages) may be exchanged among thenodes/devices of the computer network 100 using predefined networkcommunication protocols such as certain known wired protocols, wirelessprotocols (e.g., IEEE Std. 802.15.4, Wi-Fi, Bluetooth®, DECT-Ultra LowEnergy, LoRa, etc.), PLC protocols, or other shared-media protocolswhere appropriate. In this context, a protocol consists of a set ofrules defining how the nodes interact with each other.

FIG. 2 is a schematic block diagram of an example node/device 200 thatmay be used with one or more embodiments described herein, e.g., as anyof the nodes or devices shown in FIG. 1 above or described in furtherdetail below. The device 200 may comprise one or more network interfaces210 (e.g., wired, wireless, PLC, etc.), at least one processor 220, anda memory 240 interconnected by a system bus 250, as well as a powersupply 260 (e.g., battery, plug-in, etc.).

Network interface(s) 210 include the mechanical, electrical, andsignaling circuitry for communicating data over links coupled to thenetwork. The network interfaces 210 may be configured to transmit and/orreceive data using a variety of different communication protocols, suchas TCP/IP, UDP, etc. Note that the device 200 may have multipledifferent types of network connections 210, e.g., wireless andwired/physical connections, and that the view herein is merely forillustration. Also, while the network interface 210 is shown separatelyfrom power supply 260, for PLC the network interface 210 may communicatethrough the power supply 260, or may be an integral component of thepower supply. In some specific configurations the PLC signal may becoupled to the power line feeding into the power supply.

The memory 240 comprises a plurality of storage locations that areaddressable by the processor 220 and the network interfaces 210 forstoring software programs and data structures associated with theembodiments described herein. The processor 220 may comprise hardwareelements or hardware logic adapted to execute the software programs andmanipulate the data structures 245. An operating system 242, portions ofwhich are typically resident in memory 240 and executed by theprocessor, functionally organizes the device by, among other things,invoking operations in support of software processes and/or servicesexecuting on the device. These software processes/services may comprisean illustrative super-resolution process 248, as described herein. Notethat while process 248 is shown in centralized memory 240 alternativeembodiments provide for the process to be specifically operated withinthe network interface(s) 210.

It will be apparent to those skilled in the art that other processor andmemory types, including various computer-readable media, may be used tostore and execute program instructions pertaining to the techniquesdescribed herein. Also, while the description illustrates variousprocesses, it is expressly contemplated that various processes may beembodied as modules configured to operate in accordance with thetechniques herein (e.g., according to the functionality of a similarprocess). Further, while the processes have been shown separately, thoseskilled in the art will appreciate that processes may be routines ormodules within other processes.

FIG. 3 illustrates an example wireless network 300, according to variousembodiments. Wireless network 300 may be deployed to a physicallocation, such as floor 302 shown, and may include variousinfrastructure devices. These infrastructure devices may include, forexample, one or more access points (APs) 304 that provide wirelessconnectivity to the various wireless clients 306 distributed throughoutthe location. For illustrative purposes, APs 304 a-304 d and clients 306a-306 i are depicted in FIG. 3. However, as would be appreciated, awireless network deployment may include any number of APs and clients.

A network backbone 310 may interconnect APs 304 and provide a connectionbetween APs 304 and any number of supervisory devices or services thatprovide control over APs 304. For example, as shown, a wireless LANcontroller (WLC) 312 may control some or all of APs 304 a-304 d, bysetting their control parameters (e.g., max number of attached clients,channels used, wireless modes, etc.). Another supervisory service thatoversees wireless network 300 may be a monitoring and analytics service314 that measures and monitors the performance of wireless network 300and, if so configured, may also adjust the operation of wireless network300 based on the monitored performance (e.g., via WLC 312, etc.). Infurther embodiments, as detailed below, monitoring and analytics service314 may also be configured to perform object modeling from a motionand/or kinematics standpoint for purposes such as objectdetection/identification, object tracking, object behavioralpredictions, alerting, and the like. Note that service 314 may beimplemented directly on WLC 312 or may operate in conjunction therewith,in various implementations.

Network backbone 310 may further provide connectivity between theinfrastructure of the local network and a larger network, such as theInternet, a Multiprotocol Label Switching (MPLS) network, or the like.Accordingly, WLC 312 and/or monitoring and analytics service 314 may belocated on the same local network as APs 304 or, alternatively, may belocated remotely, such as in a remote datacenter, in the cloud, etc. Toprovide such connectivity, network backbone 310 may include any numberof wired connections (e.g., Ethernet, optical, etc.) and/or wirelessconnections (e.g., cellular, etc.), as well as any number of networkingdevices (e.g., routers, switches, etc.).

The types and configurations of clients 306 in network 300 can varygreatly, ranging from powerful computing devices to any number ofdifferent types of IoT nodes/devices. For example, clients 306 a-306 imay include, but are not limited to, wireless sensors, actuators,thermostats, relays, wearable electronics, and the like.

—Motion Dynamics and Inverse-Kinematics Super-Resolution with RadioSensing—

As noted above, inferring an object model and the behavior of an objectfrom wireless signals now becomes possible thanks to the proliferationof wireless communications. For example, in a simple case, radio signalsreceived by an AP 304 in wireless network 300 can be used to detect whena client 302 is approaching it (e.g., for purposes of making a physicalsecurity assessment, etc.). According to various embodiments, Bayesianinference methods are introduced herein that allow a service (e.g.,service 314) to infer the motion dynamics and/or inverse-kinematics ofan object with super-resolution, based on received wireless signals.

To illustrate the super-resolution modeling techniques introducedherein, consider the operation of the Global Positioning System (GPS).As would be appreciated, the data captured by a GPS receiver comprisestime echoes from in-sight satellites. However, these time records areuseless on their own. When combined with longitude, latitude, andelevation as state estimates, though, GPS functions as one of the mostprecise location and navigation systems ever. To perform the estimationsin GPS, a Doppler Differential Kalman Filter is used, which estimatesthe probability distribution of interest for the current stateconditioned on the readings/measurements taken prior to the currenttime.

The techniques herein, on the other hand, are directed to providing anoptimal statistical inference defined by the physics of motion as theunderlying model/prior merging the data produced by multiple streams(e.g., wireless signals from any number of devices), to derive new andunmeasured states of the modeled object characteristics. In particular,the present disclosure describes a number of techniques that apply aBayesian inference method of super-resolution to infer: a.) motiondynamics of objects and/or b.) inverse kinematics of objects, where theinferences are based on the difference of the models' state estimationsand the measured signals of any combination of either single-spectralradio signal data or multi-spectral radio signal data. Notably, theproduced inference is deemed optimal, and therefore super-resolved forthe motion. Such techniques are widely applicable, and may beparticularly well suited for health care, security, and safety, amongmany other use-cases.

Specifically, in the context of modern radio imaging, the proliferationof radio technology and devices in consumers, industry and governmentspresent a larger number and great diversity in radio emitters andsensors. The techniques herein may be applied to this growing paradigmof radio emitter and sensors through the dynamics and inverse kinematicsstate estimations of the physical states of the objects in question. Forinstance, in terms of motion dynamics, objects may be modeled to theirintrinsic elements using the Newtonian physics of motion (motiondynamics). By way of example, FIG. 4 illustrates an example human 400that can be represented by his or her center of mass (CM) 402 (e.g.,center of gravity) on which Newtonian forces apply.

The kinematics behavior of the object can similarly be modeled byapplying kinematics laws to the intrinsic element of the articulations(joints) of the object. The object's components are therefore assumed tohave a rigid shape. For example, FIG. 5 illustrates a kinematics staterepresentation of a person 500. As shown, person 500 may be representedas a series of chained points connected by rigid segments such as head502, right (R.) shoulder 504, right elbow 506, right wrist 508, left(L.) shoulder 510, left elbow 512, left wrist 514, chest 516, abdomen518, right hip 520, right knee 522, right ankle 524, left hip 526, leftknee 528, and left ankle 530. As would be appreciated, the number andspecificity of these points can vary, as desired. Each such point502-530 may have its own six degrees of freedom in the kinematics model,but each segment may be constrained by the others. For example, themovement of right knee 522 may, in turn, cause right ankle 524 to alsomove. Thus, the movement of points 502-530 may be interdependent on oneanother.

Therefore, an example use case of the techniques herein would be toinfer a human's motion dynamics (location) and inverse kinematics(physical behavior) from received radio signals. This particularexample, for instance, specifies the human constitute of multiple pointsjoints (kinematics) following the general motion dynamics. To illustratethe relationship between the motion dynamics and kinematics behavior ofan object, assume that the service uses radio signals to determine thatright hip 520 is moving in a specific direction. In turn, the servicecan leverage inverse kinematics to propagate this motion to the otherpoints 502-518 and 522-530 in a constrained and statistical sense.

Said differently, the techniques herein allow for a service, such asservice 312, to represent the motion (motion dynamics) and behavior(kinematics) operating upon data derived from radio imaging usingsuper-resolution Bayesian methods and signal processing methods to modelthe electromagnetic field variability as it interacts with the objects.Though one possible outcome of these techniques is an avatar mimicking aremote person in presence of radio signals, other outcomes may be madeto model objects using motion dynamics and inverse kinematics,accordingly. Note also that Digital Signal Processing referred to hereinpertains to the nature of the raw signal measured at the antenna capableof detecting the signal amplitudes and the phase shifts of theelectromagnetic signals.

By way of example, FIG. 6 illustrates a simplified example of APs 304 jand 304 k transmitting wireless signals 602 j and 602 k, respectively,towards person 500. In turn, each AP 304 may receive wireless signals inresponse, such as reflected signals, which the service associated withAPs 304 can use for purposes of motion and inverse kinematics analysis.

The problem of integrating information from different radio sensors toanswer particular questions is a familiar one in remote sensing, andoften is referred to as “data fusion.” “Data fusion,” as commonlydefined in the state of the art, however, is fundamentally incorrect asdata should never be tampered with, let alone “fused.” Data is what wasobserved, and as such cannot be changed after the event. In contrast,the techniques herein operate under the theory that it is possible toconstruct physical, mathematical, and behavioral models (motion andkinematics models) of the underlying physics of an object that “best”predicts the observed data.

From such physical models, it is possible to project new derived datapoints at a higher resolution and fidelity, which appears as a set ofsystems' state variables. This set is a union of state variables thatare measured and that are purely derived from physical and behavioralmodels. That is, given the model, one can probabilistically predict whatwould be observed (the expected data) and the non-observed through theunderlying models.

In general, super-resolution is an optimal method that combines thesensed data with the physics models' optimal precisions. Continuing theGPS example above, by measuring the time between the GPS receiver andthe satellites, GPS produces derived states, namely longitude, latitude,and elevation. Therefore, advanced mathematical methods are used in GPSto combine the data from multiple satellites as well the dataaccumulated over time to achieve high precision. In this case, combiningthe computed GPS state with the behavior of the physical object in termsof its motion as a function of time resolves further the actual positionas the motion characteristics (e.g., acceleration, velocity, anddisplacement) and in this case the limits of laws of physics that bindthe underlying moving object.

The extension of this analogy to motion dynamics and inverse kinematicswould be a person equipped with multiple GPS receivers located atdifferent points of their body, which would resolve further throughkinematics models of the body and better precisions of the locations ofthe joints. Inverse kinematics super-resolution is a complex computationprocess depending on the motion and kinematics models acting as a strongprior. (Notably, body models are often defined statically by the jointcomponents.)

The implication of a strong model/prior optimally eliminates noise fromdata. This is also known as the Kalman Filter. Therefore, for theanalogy example according to the techniques herein, if the person isequipped two low resolution GPS receivers at different joint location,the inference links the two GPS data through inverse kinematics andwould produce a higher resolution when compared to each independent GPSfindings.

According to various embodiments, the proposed techniques herein utilizeradio signals to infer positions of the articulation points of a modeledkinematics chained object. According to the techniques herein, an objectmay be super-resolved independently in time on its own motion as afunction. The object motion is therefore inferred and super-resolvedover time as a single dense object represented by its center masslocation the and the object's mass. In addition, according to thetechniques herein, the object kinematics may be inferred and furthersuper-resolved over time based on the inverse-kinematics as defined bythe chained objects. Further inference on the expected data computedfrom kinematics can be used to update further the model. For example,human motion kinematics has a known behavioral pattern (e.g., walking,sitting, etc.). The same inference, and when it differs from knownkinematic or established behaviors, may constitute a clue of thebehavior, to determine whether the model is trustworthy. For example,sudden acceleration of the object in the vertical direction may bepresumed to be a fall, as it does not follow the known pattern.

The motion dynamics and kinematics models act as the central repositoryof all the radio information in the data and is constructed from theprior knowledge of physical processes and how physical dynamicsinteracts with their environment. This central model could loosely bedescribed as the system that “fuses” data, but it is not itself “data”or a “data product”. The Bayesian probabilistic estimation approach usedherein not only allows estimation of the most probable model given priordomain knowledge and the data, but it also estimates the uncertaintyassociated with the model. In particular, if the model's uncertainty ishigh, it means that there is insufficient data/prior knowledge toaccurately construct the required model.

The Bayesian model-based data integration outlined above (a.k.a. Dynamicand Extended Kalman Filter), in principle solves the problem of how tointegrate information from multiple sensors, namely radio antennasincluding multi-spectral and other sensing modality.

In multi-spectral sensor data, different resolutions in wavelength willupdate the models at different resolutions, enhancing further the modelresolution. In this case, enhancing the resolution beyond the Nyquistlimit of sensor resolution is what is referred to herein as“super-resolution.” In other words, super-resolution combines optimallythe input data where the derived resolution exceeds the resolution ofthe highest sensor resolution.

Loosely said, super-resolution for motion dynamics andinverse-kinematics breaks the Nyquist limit. Single sensorsuper-resolution methods would hope to improve the resolution of theunderlying data and overcoming the Nyquist limit only when motion isestablished. However, when more sensors and data points translatedthrough their underlying physics and behavioral laws to thehigh-resolution models, high-resolution data is consequently producedwith optimal statistical definition. Simply put, super-resolution is atradeoff between the highly correlated statistical data againstmulti-dimensional interpolation and extrapolation through the underlyingphysical and behavioral models.

It is also quite useful for the service to identify changes when thedifference between what the model behavior should be (estimating motionand kinematics) and the observed data is exceeding the noise threshold.The models can also be used in a differential approach against the datawith the options to either resolve further additional models and newmodels' parameters or resolve and infer a larger parameter set (e.g.,increase the number of joints for a complex object).

According to the illustrative embodiments herein, certain mathematicsand formulas have been conceived for techniques herein, which areillustratively described below in the following Parts. In practice,uncertainty will be dominant in electromagnetic sensing, the choice ofmodels and derivation are intended to carry forward the uncertainty andapproximation errors over to the target medium models' characteristics.These uncertainties will skew the medium inferred expected parameters.Since the medium is a physical object, its parametric representationwill be of Gaussian with a mean and variance. However, the skewedness ofthe uncertainty is readily removed from the final statistics as themodel's nature are known a-priori. For example, humans may be the actualobject medium in question and not a random object and therefore themodel/prior is known to a certainty of fidelity for further inference inthe inference derivation.

Notably, before continuing further in the description, it is importantthat the following symbols and nomenclature are understood:

-   -   1. Angular frequency ω (radians/sec)=2πf, where f=frequency (Hz)    -   2. This fall-off is far more rapid than the classical radiated        far-field E and B fields, which are proportional to the        inverse-distance (1/d) where d is the Euclidean distance.    -   3. The power density Γ (Gamma) of an electromagnetic wave is        proportional to the inverse of the square of distance from        source (Γ∝1/d²).

Part I: Signal Reflection and Surface Models

When an electromagnetic field is incident upon a boundary, in general itwill split up into reflected and refracted fields. For a wave striking aseparation interface of materials, Snell's law state that the incidentangle equals the reflection angle θ1=02 such that:

ν₁ sin θ₁=ν₂ sin θ₂

with ν₁, ν₂ being the refractive indices of the two media, thereflection coefficient is:

$R_{E} = {\frac{{v_{1}\cos \; \theta_{1}} - \sqrt{v_{2}^{2} - {v_{1}^{2}\sin^{2}\theta_{1}}}}{{v_{1}\cos \; \theta_{1}} + \sqrt{v_{2}^{2} - {v_{1}^{2}\sin^{2}\theta_{1}}}}.}$

It is of interest to model the specular reflectance when theelectromagnetic field reflects over a convex closed boundary object.When the total surface of the object represents curves represented inspherical terms, the closed integral for R_(E) over the convex surfaceconverges to a Lambertian Reflectance approximation.

Lambertian reflectance is the property that defines an ideal “matte” ordiffusely reflecting surface. The apparent brightness of a Lambertiansurface to a receiver antenna is the same regardless of the antennasangle. More technically, the surface's reflectance is isotropic, and theintensity obeys Lambert's cosine law.

The Lambertian approximation is a model of the perceived reflectance ofa surface or a component of the wave intensity in a way that is as closeto reality. The Lambertian model is further validated where the closedintegral of the reflections in indoors environment includes theline-of-sight and multipaths reflections.

With further considerations that the natural random aspects andvariations of the underlying surfaces. The assumption that theelectromagnetic radio reflectance can be approximated with theLambertian's reflectance surfaces is valid choice. For example, theassumption is true for general human surfaces.

The reflection of the surface of a medium over convex surface is:

R _(E)=√{square root over (A _(e))}Φ

where Φ_(m) is infinitesimal reflected radiation flux for the Lambertianmodel for a point surface such that:

$\Phi = {\sqrt{\rho}\frac{1}{R_{s}}\left( {{E_{s}\cos \alpha^{s}} + E_{A}} \right)\cos \alpha^{r}}$

where ρ is the albedo (material reflection coefficient between 0-1) ofthe surface, E_(s) is the perceived radiation intensity direct from thesource antenna, E_(A) is the perceived ambient radiation (i.e.multipaths), and R is the distance driven diffusion factor from sourceantenna. E_(A) is also inversely proportional dependent on distance fromthe source (for non-infinite distance sources). Further, α^(s) and α^(r)are the angle from the surface norm towards the source and receiverrespectively.

A general derivation would consist of the source and receiver havingdifferent locations. For this example, it can be assumed that bothsource and receiver are collocated. The close integral for theline-of-sight and multipaths flux is approximated as follows:

$\Phi = {\sqrt{\rho}\frac{1}{R}\left( {E_{s} + E_{A}} \right)}$

To compute the Lambertian flux bidirectional reflectance distributionfunction (BRDF) for an object, the projected A_(e) is defined as theeffective surface area, which may be modeled experimentally, being half(½) of the total surface for the object. For volumetric computations, anassumption can be made that objects present complex curvatures which canbe further approximated in spherical and/or cylindrical intrinsic forms.For example, for a fractional cylindrical form as an assumption, theapproximation with a radius r and height and h is:

A _(e) =rh∫ ₀ ^(π)sin(x)dx=2rh

where r and h are considered random variables and have a Gaussians innature. When motions occur, objects will produce change in theirdisplacements of multiple random surfaces which are perceived by areceiving antenna. The receiving antenna can therefore sense anelectromagnetic signal intensity change E_(R) being the sum of thecontributions of moving surfaces represented by their respectiveeffective areas and flux. The sum of the projections is therefore:

A_(e) = rh∫₀^(π)sin (x)dx = 2  rh and$R_{E} = {\sum\limits_{i}^{N}{\sqrt{A_{ei}}\Phi_{i}}}$

With the assumption that not everything moves instantaneously, themeasurement of the signals at very high sampling rate (e.g.micro-seconds range) will isolate each motion independently as amicro-motion and for a virtual micro-surface in the Euclidean space witha respective area A_(e) and micro-displacement dλ at that moment, thedisplaced volume is therefore:

V=A _(e) dλ

The micro-mass can be also computed for an assumed mass density. Othercurvatures and shapes models can be considered such as spheres,ellipsoids instead of a cylindrical form. The mathematical derivationwill follow a similar derivation to a cylinder.

By way of example, consider the case illustrated in FIG. 7A. As shown,assume that the object has a three dimensional (3D) micro-surface 702that has an area greater than λ. During operation, both line-of-sightwaves 704 and multipath waves 706 will reflect off of micro-surface 702and may be received by an AP, such as AP 304 j shown. From this, themotion 708 (e.g., the micro-displacement dλ) of micro-surface 702 can becomputed.

Part II: Signal Attenuations

In terms of the parameters of an electromagnetic wave and the mediumthrough which it travels, the wave impedance may first be computed. Thesurface (entry) to surface (exit) wave impedance Z of the medium isdefined as the ratio of electrical and magnetic field and is evaluatedas:

$Z = {\frac{E_{A}}{H_{A}} = \sqrt{\frac{j\; \omega \; \mu}{\sigma + {j\; \omega \; ɛ}}}}$

where E_(A) is the electric field and H_(A) is the magnetic field forthe ambient radiation and have complex representation. The impedancealso is a complex number where:

-   -   μ is the magnetic permeability,    -   ε is the (real) electric permittivity,    -   σ is the electrical conductivity of the material the wave is        travelling through (corresponding to the imaginary component of        the permittivity multiplied by omega),    -   j is the imaginary unit, and    -   ω is the angular frequency of the wave.

The electromagnetic field in the far-field is independent of theantennas (e.g., APs 304, etc.). Since wave impedance is the ratio of thestrength of the electric and magnetic fields, which in the far field arein phase with each other. Thus, the far-field impedance of free space ispurely resistive and is given as:

$Z_{0} = {\frac{E}{B} = {\sqrt{\frac{\mu_{0}}{ɛ_{0}}} = \frac{1}{ɛ_{0}c}}}$

For the virtual medium positioned in the free space between the sourceand receiver, its impedance is a function of frequency. This case ismore prominent in outdoor where the effect of multipath is minimal. Thetotal impedance is therefore the total sum of impedances and:

$Z = {\frac{1}{ɛ_{0}c} + \sqrt{\frac{j\; \omega \; \mu}{\sigma + {j\; \omega \; ɛ}}}}$

When the medium is in motion, the receiver is assumed to measureconceptually a varying impedance, assuming the medium totally lieswithin the source and receiver antennas.

Part III: Measurements

Assume now that the transmitter is a uniform source and that the signalis reflected via the object micro-surface, follows Lambertianreflectance, and then arrives at the receiving antenna, as in the casein FIG. 7A. In such a case, the Lambertian reflectance model derivedpreviously for indoors may be used to model this reflectance, althoughother derivations are also contemplated for other use cases. In terms ofpower representation in a free space, attenuation (e.g., the receivedpower at the receiving antenna) is given by:

$\begin{matrix}{{P_{RX} = {\frac{A_{RX}}{4\pi R^{2}}P_{TX}}},} & (a)\end{matrix}$

where P_(TX) is the power reflected (transmitted) by the surface, andA_(RX) is the effective area of the receiving antenna and R is thedistance between that object (surface) and the receiving end. In termsof electric field strength, this yields the following:

$\begin{matrix}{{P_{RX} = {\frac{{E_{r}}^{2}}{Z_{0}}A_{RX}}},{and}} & (b) \\{{P_{TX} = {\frac{{E_{s}}^{2}}{Z_{0}}\rho \; A_{e}}},} & (c)\end{matrix}$

where |E_(r)| and |E_(s)| are the magnitudes of the electric fieldstrength at the receiving and transmitting ends, respectively. Further,A_(e) denotes the effective area of the object, Z₀ is the impedance offree space, Z₀=120πΩ, and ρ is the albedo. In other words, ρA_(e)denotes the effective micro-surface area of the object. Substitutingfrom Equations (b) and (c) into (a) yields:

${E_{R}} = {\frac{\sqrt{\rho \; A_{e}}}{2\sqrt{\pi}R}{{E_{S}}.}}$

Now, let M denote the normalized magnitude of electric field strengthsas follows:

$M = {\frac{E_{R}}{E_{s}} = {\frac{\sqrt{\rho \; A_{e}}}{2\sqrt{\pi}R}.}}$

Furthermore, the phase difference between the received and transmittedelectric field is defined as follows:

$\psi \overset{\Delta}{=}{\psi_{E_{r}} - {\psi_{E_{s}}.}}$

The measured phase is folded/wrapped due to the recurrencecharacteristic of phase. Therefore, in various embodiments, the servicemay transform the measured phase into the true, ‘unwrapped’ value. Inone embodiment, the device may unwrap the measured phase using thefollowing algorithm, which is represented in pseudocode:

Input: Measured (wrapped) phase ψ_(l) of L subcarriers 1 = 1: L. Output:Unwrapped phase ϕ₁ of L subcarriers; 1 = 1 : L. Initialize ϕ₁ = ψ₁ for 1= 2 : L do if ψ_(l) − ψ_(l−1) > π then d = d + 1 end if ϕ_(l) − ψ_(l) −2πd endfor

For example, in the case of Orthogonal Frequency Division Multiplexing(OFDM) subcarriers, the difference between the unwrapped phases can berepresented as follows:

$\varphi \overset{\Delta}{=}{{\varphi_{E_{r}} - \varphi_{E_{s}}} = {{{- \frac{2\pi}{\lambda}}2R} = {{- \frac{4\pi}{\lambda}}R}}}$

where λ is the wavelength such that

${\lambda = \frac{c}{f}},$

c is the speed of light, and f is the frequency of operation.

Part IV: Dynamic System Model and Motion

Once motion is achieved, changes in the signal strength/gain and itsphase are perceived on the receiver antenna. When consideringinfinitesimal motion, the signal phase change (phase shift) is affectedby the medium displacement travelled divided by the wavelength fornormalization for the angular notation.

For example, as shown in plot 710 in FIG. 7B, assume that the receivingantenna receives both line of sight and multipath signals. As shown,signal C is leading signals D and E, signal D is leading signal E andlagging behind signal C, and signal E is lagging behind both signals Cand D. In such a case, the service may determine both the magnitude andphase of each signal and add the signals, accordingly.

In FIG. 7C, when motion occurs, as represented by dλ, the service maydetermine the corresponding change in magnitude dM and change in phasedφ, in order to model the motion of the object. This can be performed,in some embodiments, on a per-antenna and/or per-device (AP) basis.

From the previous equations, the wave formation is a function of directand multipath reflections with the effect of multipath traversing themedium is approximated as follows:

${E_{R}} = {\frac{\sqrt{\rho \; A_{e}}}{2\sqrt{\pi}R}{E_{s}}}$

Infinitesimally, the derivative matrix (Jacobian) across multiplereceivers over the distances R and the area A_(e) where A′_(e)=ρA_(e),is therefore:

$\begin{matrix}{J = \left\lbrack {\frac{{dE}_{R}}{dR}\mspace{25mu} \frac{{dE}_{R}}{dA}} \right\rbrack} \\{= {\begin{bmatrix}{- \frac{\sqrt{A_{e}^{\prime}}}{R_{1}^{2}}} & \frac{\sqrt{A_{e}^{\prime}}}{R_{1}R_{2}} & \ldots & \frac{\sqrt{A_{e}^{\prime}}}{R_{1}R_{N}} & \frac{1}{2R_{1}\sqrt{A_{e}^{\prime}}} \\\frac{\sqrt{A_{e}^{\prime}}}{R_{2}R_{1}} & {- \frac{\sqrt{A_{e}^{\prime}}}{R_{2}^{2}}} & \ldots & \frac{\sqrt{A_{e}^{\prime}}}{R_{2}R_{N}} & \frac{1}{2R_{2}\sqrt{A_{e}^{\prime}}} \\\vdots & \vdots & \ddots & \vdots & \vdots \\\frac{\sqrt{A_{e}^{\prime}}}{R_{N}R_{1}} & \frac{\sqrt{A_{e}^{\prime}}}{R_{N}R_{2}} & \ldots & {- \frac{\sqrt{A_{e}^{\prime}}}{R_{N}^{2}}} & \frac{1}{2R_{N}\sqrt{A_{e}^{\prime}}}\end{bmatrix}.}}\end{matrix}$

One observation made herein is that the derivative is invariant of theobject shape as long the object projected A_(e) sensitivity to motion isnot marginal as to the effect of motion. Further derivation can beconstructed to model A_(e) being affected by motion when the underlyingmedium is considered non-symmetrical in shape. Note that {right arrowover (μ)}, A_(e) and velocity {right arrow over (ν)} are the dynamicmotion of a micro-object.

Part V: Minimization and Dynamic System Estimation from MultipleAntennas

The specific form of the general model parameters that are considered bythe service may be parameterized by a set of 3-dimensional parametersvector {right arrow over (u)}(t) and medium micro-area A_(e)(t) and avelocity {right arrow over (v)}(t) in a three-dimensional space asfollows:

p({right arrow over (u)},{right arrow over (v)}A _(e) |E ₁(t) . . . E_(n)(t))∝p(E ₁ . . . E _(n) |{right arrow over (u)},{right arrow over(v)},A _(e))p({right arrow over (u)},{right arrow over (v)},A _(e))

where Ê₁({right arrow over (u)},{right arrow over (v)},A_(e)) denotesthe synthesized signal from the model and σ_(e) ² is the noise variance.This gives way to a Bayesian inference model that can be solved usingthe following:

${p\left( {\left. {{E_{1}(t)}\mspace{14mu} \ldots \mspace{14mu} {E_{n}(t)}} \middle| \overset{\rightarrow}{u} \right.,\overset{\rightarrow}{y},A_{e}} \right)} \propto {\prod\limits_{i}{\exp\left( {- \frac{\sum\left( {E_{i} - {{\hat{E}}_{i}\left( {\overset{\rightarrow}{u},\overset{\rightarrow}{y},A_{e}} \right)}} \right)^{2}}{2\sigma_{e}^{2}}} \right.}}$

where Ê₁({right arrow over (u)},{right arrow over (v)},A_(e)) isestimated from the models stated above. In various embodiments, applyingBayes' theorem to the problem yields:

p(R ₁ . . . R _(n) ,A _(e) |E ₁(t) . . . E _(n)(t))∝p(E ₁(t) . . . E_(n)(t)|R ₁ . . . R _(n) ,A _(e))p(R ₁ . . . R _(n) ,A _(e))

where R₁ . . . R_(n) are the distances between object and the antennas,A_(e) is the effective area, and E₁(t) is the measured signal at thereceiving i^(th) antenna, as well as the following:

p(x,y,z,s|R ₁ . . . R _(n) ,A _(e))∝(p(u|R ₁ . . . R _(n) ,A_(e))p(x,y,z)

where x,y,z are the position of the surface (x, y, z) and are estimatedby solving the following set of quadratic equations for x, y and z andas velocities, v_(x) _(i) , v_(x) _(i) , v_(z) _(i) as follows:

${{{\left( {x - x_{i}} \right)^{2} + \left( {y - y_{i}} \right)^{2} + \left( {z - z_{i}} \right)^{2}} = {sR_{i}^{2}}}{\left( v_{x_{i}} \right)^{2} + \left( v_{y_{i}} \right)^{2} + \left( v_{z_{i}} \right)^{2}}} = \left( \frac{\frac{\lambda}{4\pi}d\varphi_{i}}{dt} \right)^{2}$

where {right arrow over (u)}(t) and {right arrow over (v)}(t) are thevector system dynamics of the location and velocity of themicro-surface, dϕ_(i) is the change in the phase measured at the i^(th)antenna, and s is a scale to accommodate for the unknown constants. Thisstates that the posterior distribution of the motion dynamics parametersand the effective area are proportional to the likelihood (e.g., theprobability of observing the data given the parameters) multiplied bythe prior distribution on the motion dynamics parameters and effectivearea.

For illustrative purposes only, the single-spectrum derivations aboveare for any numbers of antennas. However, those skilled in the art willappreciate that multiple spectral antennas derivations may be combinedin a similar fashion as an extension to the single-spectrum derivationsillustrated herein, and without departing from the intent and scope ofthe techniques embodied within the present disclosure. For example,signal data received by 2.4 GHz and 5 GHz antennas can be combined tomake a single inference for a particular object.

Updating Object Motion Dynamics from Multiple Radio Signals

The present disclosure is directed to certain aspects of the techniquesdescribed above, particularly in relation to updating object motiondynamics from multiple radio signals. Indeed, using the abovederivations and modeling approaches detailed above, a service for awireless network may infer and update the various motion dynamics of a

Illustratively, the techniques described herein may be performed byhardware, software, and/or firmware, such as in accordance with theillustrative super-resolution process 248, which may include computerexecutable instructions executed by the processor 220 (or independentprocessor of interfaces 210) to perform functions relating to thetechniques described herein (e.g., to provide a super-resolution serviceto a network).

Specifically, according to various embodiments, a service receivessignal characteristic data indicative of characteristics of wirelesssignals received by one or more antennas located in a particular area.The service uses the received signal characteristic data as input to aBayesian inference model to predict physical states of an object locatedin the particular area. A physical state of the object is indicative ofat least one of: a mass, a velocity, an acceleration, a surface area, ora location of the object. The service updates the Bayesian inferencemodel based in part on the predicted state of the object and a change inthe received signal characteristic data and based in part by enforcingNewtonian motion dynamics on the predicted physical states.

Operationally, in various embodiments, the above techniques can beaugmented by the super-resolution service/process enforcing the laws ofmotion dynamics on the motion of a detected body/object. Indeed, whenusing a Bayesian inference model to infer the physical state(s) of anobject based on wireless signals received by any number of antennas, theinferred physical state(s) should still conform to the laws of Newtoniandynamics. Accordingly, in various embodiments, the Bayesian inferencemodel may also take into account these dynamics laws, when making itspredictions. For example, in one embodiment, the computed error for themodel may also be based on whether the predicted object state isconsistent with these laws.

As would be appreciated, the dynamics laws that the service may enforceare a function of the specific state parameter(s) predicted by theinference model, such as the location of the object, the mass of theobject, the velocity of the object, and/or the acceleration of theobject. Non-limiting examples of these constrains may include, but arenot limited to, any or all of the following:

TABLE 1 Predicted Object State Parameter Constraint Surface Area Thepredicted surface area must be consistent with the estimated distancesof the object to the signal receivers (e.g., APs) and motion of theobject. Location The predicted location must be consistent with thepreviously-determined location of the object and/or the other predictedstate parameter(s). For example, once the object appears within the beamof an antenna, the object cannot simply appear or disappear. Inaddition, the existence of the object in the beam must obey an entry andexit point from the beam. The predicted location of the object shouldalso be consistent with its previous location and its velocity and/oracceleration. Mass The predicted mass of the object must be consistentwith any other predicted state parameters. For example, the linearmomentum of the object (e.g., Mass × Velocity) should be consistent withthe force acted on the object (e.g., Mass × Acceleration), etc. VelocityThe predicted velocity of the object must also be consistent with anyother predicted state parameters. For example, the predicted velocityshould be consistent with the predicted change in location of the objectover time, the predicted acceleration of the object, the kinetic energyof the object (e.g., Mass × Velocity), the direction of motion of theobject, etc. Acceleration The predicted acceleration of the objectshould also be consistent with any other predicted state parameters, aswell as any expected values. For example, if the object is falling, thepredicted acceleration may be constrained to not exceed that ofgravitational acceleration (e.g., 9.8 m/s²).

In addition to enforcing Newtonian motion dynamics on the predictedobject states, the service may also resolve the estimates of eachindividual body/object using the motion dynamics and measured datacomputed from the change in gain in the antenna(s), the phase shift rateof change, and/or the ratio of the antenna change(s) in gain over theantenna phase shift rate(s) of change.

In one embodiment, the service may employ a Kalman filter tocontinuously update the model. For example, such a filter may update themodel parameters (e.g., mass, location/position, velocity, acceleration,etc.) in real time by continuously solving for the error between themotion dynamics prediction and the change in gain in the antennas, thephase shift rate of change, and ratio of the antenna change in gain overthe antenna phase shift rate of change.

FIG. 8 illustrates an example Bayesian framework 800 for inferringobject dynamics from multiple radio signals, according to variousembodiments. For example, super-resolution process 248 may provide aservice to the network by implementing Bayesian framework 800 (e.g., anon-linear Kalman filter or the like), to infer the characteristics of agiven object, such as its motion dynamics.

As shown, framework 800 may include a Bayesian model 806 of the formdescribed in Part V above. During operation, a simulation 808 may beperformed using the prior of model 806, to form synthetic data 810.Synthetic data 810 is compared with the measured data 802 (e.g., thecharacteristics of the received signals), to form an inverse estimate804. In turn, inverse estimate 804 is used to update model 806, allowingfor repeated convergence over time.

In a further embodiment, the service could also identify the motiondynamics of the object/body using a digital twin of the assumed objectin the real world. For example, if the service determines that a humanis likely present within the area of interest, based on the receivedwireless signals, the service could create a digital twin of the personand ensure that any predictions are consistent with the expectedmovement of the digital twin.

Two example embodiments for which the techniques herein may be used,among any number of others, are the deriving, from radio signals: a.)avatar kinematics articulations and visualizations, and b.) objectbehavior classifications for object kinematics.

Regarding the first example, avatar kinematics articulations andvisualizations derived from radio signals, avatars are digital modelswith multiple possibilities of graphical representations of people andobjects and also rendering at large-scale mimicking the physicalreality. Avatars are known in computing and especially in gaming.Conceptually, avatar models are interactive to input that is eithercontrolled through user interaction interfaces such as keyboards oradvanced haptics, or automated through computer simulations and otherreasoning capabilities.

The example of the present disclosure stipulates that the interfaces ofthe avatars accept input from digital models in 2-Dimensions and3-Dimensions, with physical movement in the dimension of time. Thesemodels are to obey the law of physics dynamics of motion and kinematicsof the linked components of the models.

The techniques herein further use the methods of Bayesiansuper-resolution inference to assert, create, and update the fidelityand resolution of the avatars' models in real time of the people orobjects. Avatar resolution is enhanced overtime as the continuousreal-world measurement are processed through radio frequency signalswhen combined with the intrinsic simulations of already inferred models.These simulations constitute physics simulation of a free body motiondynamics and kinematics once the model type (e.g., human, dog, etc.).

For example, once a body is assumed to be moving horizontally, the lawof kinetics energy and motion dynamics are used to extrapolate the nextinstant which includes locations, velocities, and accelerations whichare rendered through the avatar. It is understood that the mentionedestimated physical quantities are in plural, to reflect the estimatedquantities of all linked components of a body (e.g., hand, foot, head,etc.).

The avatars reflect the super-resolved estimated physical quantitiesbased on the optimal calculations taking in consideration extrapolationcalculations (projection) with the estimation of the models' errors andthe radio frequency (RF) sensing data with the estimation of sensingerrors.

Avatars models follow known physical models with connected chains ofcomponents with known range of behavior. For example, if the avatar is ahuman type, the kinematics of the human body kinematics is known andestablished as a model/prior by the components characteristics as wellas the limitation that the articulation points offer to the movablecomponent relative to what it is connected to. For example, thekinematics of a moving finger has a known range, whereas the finger isconnected to a wrist with a known range whereas the wrist is connectedto forearm with known range, etc.

Super-resolution with an avatar in context takes two orthogonaldirections. For example, assuming the avatar is human, sensing data overtime can articulate the motion of the avatar in a direction such thatadditional the number of the articulations points of the avatar areidentified over time as well as the physical dimensions of thecomponents constituting the avatar. Once avatars are established andsimulated at large scale such as an industrial environment, the modelsare continuously inferred and updated with the radio signals at hand.Avatars states may also be further resolved through computer simulationconstrained by the physical setting they are located in.

Originally, an avatar is assigned a mass and volume with a wide errormargin. with time, the estimation of the mass and volume become moreprecise for the rendition and simulation as well interaction with theenvironment. For example, if the setting is an industrial manufacturingsetting and the avatars represent the workers, the motion of the avatarsis computed against the constraining environment such as walkinghorizontally, cannot traverse walls but must enter through doors, etc.These constraints are important to the inference to furthersuper-resolve and remove the errors made in the inference due to poor ornoisy data.

The avatar interaction with the environment constitute a feedback to theinference from the stage setting. Any failure in the inference engineusing data while interacting the models reflects two possibilities:Either the data is poor to use, or the models are not correct to startwith. These scenarios are flagged for correction. Either better antennasare place or the environment models are corrected if defects are found.

The advantage of avatars kinematics, and therefore visualization, offeradvanced safety to many industrial and health care verticals. Industrialsafety, for example, may be used to safeguard people from hazardousequipment. Health care verticals offer the benefit of observing patientsin hospital settings. (Many other use cases may be presented, and thetechniques herein are not limited to those examples specifically pointedout herein.) Many existing solutions in safety and surveillance areinvasive in nature. Camera systems are known to invade privacy.Reconstructed avatars, on the other hand, offer a simulated behaviorderiving from radio signals protecting privacy to the extent of theobject or the individual's physical characteristics such as height, bodymass index, and an approximate body composition.

Regarding the second example, object behavior classifications for objectkinematics derived from radio signals, object behavior classificationsfollows the object of kinematics modeling and simulation. Objectbehaviors are mostly known and there are always unknown patterns. Forexample, if the body is human and the kinematics motion shows a suddendrop in center of gravity of the human (sudden acceleration anddeceleration) into a sudden stop, such behavior maybe be labeled as a“fall”. The variety of the behavior classifications are wide to themotion kinematics of the body limited to the resolution made availablethrough the inference. These behaviors could be organized in anyarbitrary taxonomies such as “exercising” being a superset of “running”being a super set of the center of gravity displacement larger than 2.5mph in combination of periodic oscillation of the center of gravity inthe vertical direction over 3 inches. Object behavior classificationsare therefore pattern matching of known behavior against the kinematicsof the inferred models.

A prototype system was constructed using the techniques herein and usedto make various inferences regarding an object in a wireless network.More specifically, data was collected from a plurality of APs operatingon the 5 GHz frequency band. The three APs were arranged in a trianglewith the first AP assumed to be the origin. The second AP was alignedwith the first AP in the y-direction and located two meters away fromthe first AP in the x-direction. Finally, the third AP was located twometers away from the first AP in both the x-direction and y-direction.The object/client was located within the formed triangle and pinged theAPs at an interval with a mean of two milliseconds and a variance of onemillisecond. The received channel state information (CSI) was thenfiltered based on the MAC address of the client. During testing,multiple people were moving casually around the three APs.

FIGS. 9A-9C illustrate various examples of inferences made about theclient during testing, according to various embodiments. Plot 900 inFIG. 9A shows the estimated distances R₁, R₂, and R₃ from the three APs,respectively, to the effective moving surface of the client. Based onthese distances and the computations detailed above, the x-y positionsof the client were then inferred, as shown in plot 910 in FIG. 9B. Plot920 in FIG. 9C shows the effective surface area of the moving surfaceversus the samples. As would be appreciated, the effective surface areareflects the size of the moving surface.

FIG. 10 illustrates an example simplified procedure for updating objectmotion dynamics based on the characteristics of received wirelesssignals, in accordance with one or more embodiments described herein.For example, a non-generic, specifically configured device (e.g., device200) may perform procedure 1000 by executing stored instructions (e.g.,process 248), to provide a service to one or more networks. Theprocedure 1000 may start at step 1005, and continues to step 1010,where, as described in greater detail above, the service may receivesignal characteristic data indicative of characteristics of wirelesssignals received by one or more antennas located in a particular area.In one embodiment, the one or more antennas are wireless access pointantennas in a wireless network. In a further embodiment, the wirelesssignals may be received by a plurality of different antennas. In yetanother embodiment, the characteristics of the wireless signal compriseone or more of: a change in gain of the wireless signals, a phase shiftrate of change of the wireless signals, or a ratio of the change in gainover the phase shift rate of change of the wireless signals.

At step 1015, as detailed above, the service may use the received signalcharacteristic data as input to a Bayesian inference model to predictphysical states of an object located in the particular area. In variousembodiments, the physical state of the object may be indicative of atleast one of: a mass, a velocity, an acceleration, or a location of theobject. Notably, a Bayesian inference model can be used to predict thedynamics of an object present within the beam(s) of any number ofantennas with super resolution.

At step 1020, the service may update the Bayesian inference model basedin part on the predicted state of the object and a change in thereceived signal characteristic data and based in part by enforcingNewtonian motion dynamics on the predicted physical states. In variousembodiments, the service may do so using a Kalman filter to solve for anerror between the predicted physical states of the object and thecharacteristics of the wireless signals. In general, the service mayenforce the motion dynamics on the predicted physical states by ensuringthat any predicted physical states of an object (e.g., its location,mass, velocity, acceleration, etc.) are consistent with one anotherand/or any prior states of the object. For example, the service mayensure that a predicted location of the object is consistent with itsprevious location(s), such as by ensuring that the object does notsuddenly disappear, the object has an associated entry and/or exit pointwithin the beam, etc. In another example, if the service predicts boththe mass and acceleration of the object, the service may also ensurethat these predictions are consistent with one another according totheir Newtonian relationship, e.g., force=mass×acceleration. Procedure1000 then ends at step 1025.

It should be noted that while certain steps within procedure 1000 may beoptional as described above, the steps shown in FIG. 10 are merelyexamples for illustration, and certain other steps may be included orexcluded as desired. Further, while a particular order of the steps isshown, this ordering is merely illustrative, and any suitablearrangement of the steps may be utilized without departing from thescope of the embodiments herein.

The techniques described herein, therefore, provide for updating objectmotion dynamics from multiple radio signals. Notably, the technologyherein, as it relates generally to state estimation for motion dynamicsand inverse kinematics, addresses the use of single or multi-spectralradio signal data integration for businesses such as healthcare (e.g.,end-to-end monitoring in healthcare, such as for in-patient andoutpatient “man-down” monitoring) as well as physical security (e.g.,tracking of moving objects without camera-based privacy concerns). Inaddition, the techniques herein allow for the determination ofmicro-motions of an object, localizations of the object (e.g., the x-ycoordinates of the object in the area), and clustering of thelocalizations for a given time window (e.g., to assess the spreadfunction of the coordinates). Such computations can be used, forexample, to generate 3-D or 2-D renderings of the object, forming aheatmap of potential locations of the object, and the like.

While there have been shown and described illustrative embodiments thatare directed to particular embodiments and features of motion dynamicsand inverse-kinematics super-resolution with radio sensing, it is to beunderstood that various other adaptations and modifications may be madewithin the intent and scope of the embodiments herein. For example,while certain implementations are disclosed herein, such as avatars,human body sensing, access point sensors, and so on, the techniquesherein are not limited as such and can be applied to any number ofdifferent types of arrangements and/or verticals. Further, while certainspecific formulas have been shown and described, such formulas may bemodified or presented in a different manner, and the techniques hereinare not so limited.

The foregoing description has been directed to specific embodiments. Itwill be apparent, however, that other variations and modifications maybe made to the described embodiments, with the attainment of some or allof their advantages. For instance, it is expressly contemplated that thecomponents and/or elements described herein can be implemented assoftware being stored on a tangible (non-transitory) computer-readablemedium (e.g., disks/CDs/RAM/EEPROM/etc.) having program instructionsexecuting on a computer, hardware, firmware, or a combination thereof.Accordingly, this description is to be taken only by way of example andnot to otherwise limit the scope of the embodiments herein. Therefore,it is the object of the appended claims to cover all such variations andmodifications as come within the true intent and scope of theembodiments herein.

What is claimed is:
 1. A method comprising: receiving, at a service,signal characteristic data indicative of characteristics of wirelesssignals received by one or more antennas located in a particular area;using, by the service, the received signal characteristic data as inputto a Bayesian inference model to predict physical states of an objectlocated in the particular area, wherein a physical state of the objectis indicative of at least one of: a mass, a velocity, an acceleration, asurface area, or a location of the object; updating, by the service, theBayesian inference model based in part on the predicted state of theobject and a change in the received signal characteristic data and basedin part by enforcing Newtonian motion dynamics on the predicted physicalstates.
 2. The method as in claim 1, wherein the one or more antennasare wireless access point antennas in a wireless network.
 3. The methodas in claim 1, wherein the characteristics of the wireless signalcomprise one or more of: a change in gain of the wireless signals, aphase shift rate of change of the wireless signals, or a ratio of thechange in gain over the phase shift rate of change of the wirelesssignals.
 4. The method as in claim 1, wherein updating the Bayesianinference model further comprises: using a Kalman filter to solve for anerror between the predicted physical states of the object and thecharacteristics of the wireless signals.
 5. The method as in claim 1,wherein enforcing Newtonian motion dynamics on the predicted physicalstates comprises: ensuring that a predicted location of the object isconsistent with a previous location of the object.
 6. The method as inclaim 1, wherein enforcing Newtonian motion dynamics on the predictedphysical states comprises: ensuring that a predicted mass and apredicted acceleration of the object are consistent with one another. 7.The method as in claim 1, wherein the wireless signals are received by aplurality of different antennas.
 8. The method as in claim 1, furthercomprising: determining, by the service and based on the predictedphysical states of the object, that the object is falling.
 9. Anapparatus, comprising: one or more network interfaces to communicatewith one or more networks; a processor coupled to the network interfacesand configured to execute one or more processes; and a memory configuredto store a process executable by the processor, the process whenexecuted configured to: receive signal characteristic data indicative ofcharacteristics of wireless signals received by one or more antennaslocated in a particular area; use the received signal characteristicdata as input to a Bayesian inference model to predict physical statesof an object located in the particular area, wherein a physical state ofthe object is indicative of at least one of: a mass, a velocity, anacceleration, a surface area, or a location of the object; update theBayesian inference model based in part on the predicted state of theobject and a change in the received signal characteristic data and basedin part by enforcing Newtonian motion dynamics on the predicted physicalstates.
 10. The apparatus as in claim 9, wherein the one or moreantennas are wireless access point antennas in a wireless network. 11.The apparatus as in claim 9, wherein the characteristics of the wirelesssignal comprise one or more of: a change in gain of the wirelesssignals, a phase shift rate of change of the wireless signals, or aratio of the change in gain over the phase shift rate of change of thewireless signals.
 12. The apparatus as in claim 9, wherein the apparatusupdates the Bayesian inference model further by: using a Kalman filterto solve for an error between the predicted physical states of theobject and the characteristics of the wireless signals.
 13. Theapparatus as in claim 9, wherein the apparatus enforces Newtonian motiondynamics on the predicted physical states by: ensuring that a predictedlocation of the object is consistent with a previous location of theobject.
 14. The apparatus as in claim 9, wherein the apparatus enforcesNewtonian motion dynamics on the predicted physical states by: ensuringthat a predicted mass and a predicted acceleration of the object areconsistent with one another.
 15. The apparatus as in claim 9, whereinthe wireless signals are received by a plurality of different antennas.16. The apparatus as in claim 9, wherein the process when executed isfurther configured to: determine, based on the predicted physical statesof the object, that the object is falling.
 17. A tangible,non-transitory, computer-readable medium storing program instructionsthat cause a device to execute a process comprising: receiving signalcharacteristic data indicative of characteristics of wireless signalsreceived by one or more antennas located in a particular area; using thereceived signal characteristic data as input to a Bayesian inferencemodel to predict physical states of an object located in the particulararea, wherein a physical state of the object is indicative of at leastone of: a mass, a velocity, an acceleration, a surface area, or alocation of the object; updating the Bayesian inference model based inpart on the predicted state of the object and a change in the receivedsignal characteristic data and based in part by enforcing Newtonianmotion dynamics on the predicted physical states.
 18. Thecomputer-readable medium as in claim 17, wherein the one or moreantennas are wireless access point antennas in a wireless network. 19.The computer-readable medium as in claim 17, wherein the characteristicsof the wireless signal comprise one or more of: a change in gain of thewireless signals, a phase shift rate of change of the wireless signals,or a ratio of the change in gain over the phase shift rate of change ofthe wireless signals.
 20. The computer-readable medium as in claim 17,wherein updating the Bayesian inference model further comprises: using aKalman filter to solve for an error between the predicted physicalstates of the object and the characteristics of the wireless signals.